It is often beneficial to generate reduced-resolution versions of images that were captured at high resolution. Many digital still capture devices are able to capture images with resolutions of 10 to 30 million pixels, and some can capture images at even higher resolutions. However, not all applications may benefit from such high image resolution, and high-resolution images require significant computing resources for transmission and for storage.
For example, video capture is an application that, in general, does not require the same resolution as is available from still cameras. 1080p high-definition video, which is the highest-resolution video format generally available in the United States, requires 2,073,600 pixels per frame. Thus video capture requires less than one tenth of the pixel resolution that is available in many current digital still capture devices. It is well known to generate reduced-resolution versions of captured images, for example to allow use of captured images in video sequences.
The process of generating reduced-resolution versions of captured images is referred to herein as “downsampling”. Downsampling may be implemented at any point in the post-capture image processing chain, either at the image capture device (camera) or at some other device.
Downsampling may be implemented, for example, as a two-step process wherein the image is prefiltered and then subsampled. Alternatively, downsampling can be implemented in a single step, wherein the downsampled image is reconstructed from the high-resolution image. In general, in either technique, each pixel in the resulting downsampled image has a value that is determined by a small group of pixels adjacent to the corresponding location in the original high-resolution image. Conventionally, downsampling is performed by reducing a group of adjacent pixels in the original image into a single pixel in the downsampled image. For example, an image can be downsampled by a factor of two in both spatial dimensions by computing each output pixel of the downsampled image as the average (or sum) of a correspondingly positioned 2×2 quadrilateral of input pixels of the original image. In general, the pixels that are used in generating a single output pixel are adjacent to one another in the original image.
Light field photography captures information about the direction of light as it arrives at a sensor within a data acquisition device such as a light field camera. Such light field data can be used, along with captured image data, to create representations of scenes that can be manipulated by a user. Subsequent to image capture, light field processing can be used to generate images using the captured image data along with the light field data. Various types of light field processing can be performed, including for example refocusing, aberration correction, changing the viewing perspective, and the like.
Existing techniques for downsampling images are, in general, not well suited to light field images. In light field images, pixels corresponding to a particular image area are not necessarily located adjacent to one another in the sensor array. As is described in related U.S. Utility application Ser. No. 12/703,367, referenced above, light field cameras include a microlens array that redirects light to individual elements of the image sensor based on the direction in which the light rays arrive at the microlens array. Thus a given image element (such as a point or region within a subject being photographed) may be represented by many disparately located pixels in the sensor array. The pixels representing the image element are, in effect, scattered throughout the sensor array by the microlenses in the microlens array. Conventional downsampling techniques are, in general, unable to combine these disparately located pixels in the sensor array. Thus, such techniques are, in general, unable to provide effective downsampled images that preserve the advantages and characteristics of the light field image, such as the ability to refocus or otherwise manipulate the image after it has been captured.